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ABSTRACT 



This study in New Zealand investigated (1) how well 
government -funded Learning Experiences Outside The Classroom (LEOTC) 
providers were meeting requirements of their contracts regarding curriculum 
linkage, school liaison, and teacher support and (2) how LEOTC providers went 
about achieving these objectives and how well their strategies worked. The 
LEOTC programs provided out -of -class programs and teacher workshops and 
developed supporting educational resource materials to help teachers with 
curriculum delivery. The research looked at whether schools perceived the 
LEOTC programs as meeting their learning needs as expressed in the curriculum 
documents and what types of schools participated in LEOTC programs. Data 
collection included a telephone survey of 40 schools participating in 
programs offered by 10 providers and an examination of relevant documents 
from 5 providers in the Auckland area. Data analysis indicated that LEOTC 
programs were playing an essential role in complementing the classroom work 
of teachers. Teachers considered them very valuable, especially in how they 
provided opportunities for hands-on experience. Providers were able to adapt 
the programs to meet the needs of diverse students. Some rural schools 
expressed a need for a wider range of programs. Teachers wanted to see more 
information about programs. All types of schools participated in the 
programs, regardless of socioeconomic status or geographic location. An 
appendix presents the telephone survey questionnaire. (SM) 
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PREFACE 



T he Curriculum Division of the Ministry of Education commissioned this small study to 
collect some independent information on schools’ perceptions of the work carried out by 
a sample of Government-funded LEOTC providers. This study has investigated the perceptions 
of some schools using the LEOTC. Time constraints on the production of this report did not 
permit a research design which included interviews with teachers at schools who had not accessed 
the LEOTC programmes in the sample. 



EXECUTIVE SUMMARY 

Aims 

* *his study aimed to provide information about: 

• how well LEOTC providers are meeting the requirements of their contracts with regard 
to curriculum linkage, school liaison, and teacher support; and 

• how LEOTC providers have gone about achieving these objectives and how well the 
strategies work. 

The research questions ask: 

• Do schools perceive the LEOTC programmes meeting the learning needs of schools as expressed 
in the curriculum documents? 

• What types of schools— that is, decile, class level, geographic location — are participating in 
LEOTC programmes} 

Methodology 

The first research question about the schools’ perception of the programmes was addressed 
through a telephone survey of 40 schools which had participated in the programmes offered by 
10 providers. The schools were chosen in order to cover a range of decile groups, levels of school, 
and geographic location. The providers were chosen in order to cover a range of geographic 
locations, including rural, regional, and urban areas, and a range of curriculum and topic areas. 
An examination of relevant documents from the 5 providers in the Auckland region was carried 
out to answer the second research question. 

Results 

The results of the 2 aspects of the study show that LEOTC programmes are playing an essential 
role in complementing the classroom work of the teachers. The teachers report that they find 
them extremely valuable, especially in the way that they afford opportunities for “hands-on” 
experience. The programmes are planned with curriculum units in mind. It is noteworthy that 
the providers were able to adapt the programmes in order to meet, for example, the special needs 
of children with disabilities, or the needs of those with English as a second language. All types 
of schools participate in the programmes, regardless of school decile ranking, or location. 

The few criticisms were very specific, and did not detract from the overall worth of the 
programmes. A number of schools, especially those in rural areas, expressed a need for a wider 
range of programmes, including more for younger children, and more that were brought to the 
schools. 

If the provision were to be expanded, there appears to be a need for more programmes in the 
curriculum areas of language, technology, and mathematics. Teachers would also appreciate 
receiving more information about the programmes, possibly in the form of a catalogue produced 
annually by the Ministry of Education. 
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EVALUATION OF LEARNING EXPERIENCES 
OUTSIDE THE CLASSROOM (LEOTC) PROGRAMME 

Introduction 

| his study aims to provide information about: 

• how well LEOTC providers are meeting the requirements of their contracts with regard 
to curriculum linkage, school liaison, and teacher support; and 

• how LEOTC providers have gone about achieving these objectives and how well the 
strategies work. 

The research questions ask: 

• Do schools perceive the LEOTC programmes meeting the learning needs of schools as expressed 
in the curriculum documents ? 

• What types of schools, that is, decile, class level, geographic location, are participating in LEOTC 

programmes} 

Background 

The LEOTC programmes have operated since 1995. They provide curriculum-based programmes 
that complement and support students classroom learning. They replace the previous situation 
where a relatively small number of providers were funded to run educational programmes for 
schools. The LEOTC programmes cover a wider range of providers and programmes. 

The primary functions of LEOTC are to provide out-of-class programmes and teacher 
workshops, and to develop supporting educational resource materials to assist teachers in the 
compulsory school sector with their curriculum delivery. A premium is placed on activities that 
are “hands-on” and interactive. 

LEOTC funds are disbursed from Vote: Education and Vote: Research, Science and 
Technology through an annual contestable round, where organisations that wish to provide 
LEOTC services for the following year submit proposals to the Ministry of Education for 
consideration. Selections are made on the basis of established criteria. The proposals must 
demonstrate the relevance of programmes to the New Zealand Curriculum Framework, describe 
the qualifications and relevant experience of staff, and display an understanding of the principles 
of learning and teaching. The programmes are expected to be developed in consultation with the 
education community. 

There are currently 35 organisations which provide LEOTC programmes. The Ministry 
states that it aims to make the programmes accessible to students in as many geographic areas as 
possible. 
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The Royal Society of New Zealand (1997) has carried out an analysis of the LEOTC files in 
order to assess how LEOTC funding promotes science and technology education. The report 
concludes that: 



The LEOTC fund provides effective support for the promotion of science and technology, and 
science and technology education. 

Methodology 

The project comprises an evaluation of a sample of the data provided by the providers of LEOTC 
programmes as part of their reporting requirements to the Ministry of Education. All providers 
supply regular milestone reports, as well as a final report at the conclusion of their contract with 
the Ministry. The evaluation of the milestone information was supplemented by material from 
a telephone survey of a sample of schools which had participated in LEOTC programmes, in 
order to examine the effectiveness of the LEOTC programme. 

The first research question about the schools’ perception of the programmes was addressed 
through the telephone survey. An examination of relevant documents from a sample of providers 
was carried out to answer the second question. 

Sample 

The sample for the telephone survey comprised 40 schools which had participated in the 
programmes offered by 10 providers. The providers were chosen in order to cover a range of 
geographic locations, including rural, regional, and urban areas, and a range of curriculum and 
topic areas. They included museums, art galleries, a zoo, an observatory, and other science 
programmes, 2 of which were also used for outdoor education programmes. The schools were 
chosen in order to cover a range of decile groups, levels of school, and geographic location. 

The sample for the document analysis was the Auckland region, where 5 providers had 
planned to work with 740 target schools (not all of the schools had been covered at the time of 
the data examination). Auckland was chosen since it had the highest number of providers and 
the greatest coverage of different types of schools. 

Data Collection and Analysis 

For the telephone survey, the researcher interviewed the teacher who had liaised with the 
LEOTC programme provider. Teachers were asked how closely the programme fitted into the 
curriculum and how well the programme was carried out. They were also asked if they were 
satisfied with their access to LEOTC progr amm e 

The most recent milestone documents from each of the 5 Auckland providers were examined 
in order to determine what sort of schools were participating in the programmes. They were 
analysed by SES decile, type of school, geographic location, and lists of curriculum areas. It was 
not possible to include the class levels covered by the programmes, since that information was not 
included in the data. 
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THE TELEPHONE SURVEY 



Characteristics of the Schools 

H alf of the 40 schools selected were from the wider urban areas covered by the 5 main cities 
(Auckland, Hamilton, Wellington, Christchurch, and Dunedin). Of the others, 9 were 
from provincial centres, 7 from small towns, and 4 small rural primary schools. They included 
primary, intermediate, and secondary schools, and one area school. They also covered the range 
of socioeconomic deciles as shown by figure 1. 

The schools were selected in the hope of covering a range of classes. However, it was not clear 



Figure 1: Decile Rankings of Schools 




1 23456789 10 

Decile 



from the information given to the Ministry by the providers which classes from the schools 
participated in the programmes. Figure 2 shows that students from all years took part in 
programmes with the 10 LEOTC providers, with the intermediate years and year 13 the most 
frequent participants. The lesser usage in years 11-12 could be attributed to the pressures of 
external examinations. Some of the schools sent groups of classes to the programmes. Three 
small rural schools sent their years 1-4, 1-6, and 4-8 to the programmes in order to economise 
on transport. A city school sent years 9-13, which was a vertical form comprising students with 
English as a second language. One of the other classes was a special needs class and another was 
an enrichment group. 
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Year 

Curriculum Areas Covered by the Programmes 

The providers covered a wide range of curriculum areas, more than were obvious from the titles 
of the programmes, or from the functions of the institution. A museum and science centre, for 
example, had programmes in areas of science, social sciences, language, and technology. All of 
the providers catered for the full range of age groups, although a few of the junior primary 
teachers commented that the discussion was difficult for their students to follow. 

Science 

Science was the most frequently covered curriculum area, and the providers covered a wide range 
of the science curriculum for half of the schools in the sample. The topics ranged from 
astrophysics and biology for the seventh formers; earth science, the living world, and electricity 
for the junior high school; metamorphosis, and a pond study for intermediate students; to marine 
life, the animal world, and earth and beyond for primary school students. One provider had 
modified a programme about the senses for a class of intermediate-age special needs students. 



Art 

About a third of the classes which visited providers participated in art programmes. The senior 
high school students took part in practical art programmes, as well as covering aspects of art 
history. Art appreciation was covered for the junior high and the intermediate schools. The 
latter also attended photography and painting programmes, along with the primary students. 



Social Sciences 

The social sciences were the third most widely covered area, especially for junior high school 
students. Programmes for them covered Maori heritage, and various aspects of change (historical 
differences in people’s lives). Change, as depicted through New Zealand history, was the main 
topic studied by the intermediate and primary students. 
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Language 



Two secondary school ESOL (English as a second or other language) groups were specially catered 
for through programmes which also covered social science areas. An intermediate and a primary 
class learnt about the links between poetry and painting. 

Health and Physical Wellbeing 

Two of the providers had programmes covering aspects of this topic. Two seventh form (year 
13) groups attended leadership camps, and a junior secondary group attended an outdoor 
education course. 

Technology and Mathematics 

One junior high and one intermediate class participated in technology programmes. Only one 
class, from an intermediate school, took part in a programme which fitted in with the math- 
ematics curriculum. 
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INPUT BY THE PROVIDER 



Before the Visit 

AA ost of the schools had been sent general information by the providers, explaining what 
J-VXservices were available. Arrangements for the visit were then made with the provider’s 
education officer, usually by telephone. Nine teachers met the officers, and some had a preview 
of the exhibition so that they could decide what was relevant for their classes. These personal 
meetings usually resulted in more detailed planning of the visit than that which resulted from the 
phone calls. Almost all of the providers then sent additional information, such as a kit, suggested 
reading, and preparatory worksheets. One teacher commented that she “didn’t send for the extra 
information, as it cost $15.30 and they wouldn’t let her preview it”. Only one teacher had had 
no prior communication. 



During the Visit 

Almost half of the visits included an oral presentation by the education officer, usually with the 
aid of slides, charts, overhead projections, or videotapes. Most providers allowed time for 
discussion and questions. Then the general pattern of the programmes was that the students were 
divided into small groups, either with a provider staff member with each group, or with the staff 
circulating among the groups. The exhibitions almost always included “hands-on” experience for 
the students, for example, looking at slides in a laboratory, practical art, and activating exhibits 
in a museum: 

They arranged for us to do things in small groups. The children made cream and pikelets, 
put washing through a wringer and hung it up with wooden pegs, “shaved” with a cut-throat 
razor. They also had a classroom session with a teacher with a cane, doing “chalk and talk”. 
(Museum visit, years 1-4, rural school) 

(The students) could use the equipment, doing stencil work, overlaying. (Art Gallery years 
7-8) 

Flower lesson inside, pictures and talks . . . Assembled flowers using wood and cardboard. 

Made perfume . . . Outside, looked at flowers, took them apart, pasted them on cardboard. 

Looked at how seeds grow, using actual plants at different stages of the process. Then each 
was given a pot, and some seeds to take home. (Science visit, special needs class) 

During the 3 extended visits for outdoor education/leadership experiences, the provider staff 
organised the programmes, and had a great deal of input into all aspects of the programme. Two 

had the local twi joining with provider staff to ensure that the students had an authentic cultural 
experience. 
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After the Visit 



Most follow-ups comprised evaluation forms for the teachers. Some groups were given additional 
information, or follow-up activities were suggested, and some students were given resources or 
samples. One provider telephoned the teacher to check on the success of the trip. 

Over half of the teachers said that there had been no follow-up instigated by the provider. 



SIGNIFICANCE OF THE VISIT 



Importance of the Visit for the Coverage of the Unit/Module 

| | n ly 8 of the 40 respondents said that the visit was not very important and that they could 
V_y have taught the module without the trip. The other teachers said either that they could not 
have managed without it, or that they could have managed but their coverage would probably 
not have been as comprehensive. 

Positive Comments 

Most of the teachers said that the opportunity for the students to have “hands-on” experience, or 
to actually see the objects of their study, was irreplaceable. Their school did not have the 
resources, mainly because they could not afford them. It often would be impracticable for the 
school to attempt to reproduce the programme. For some teachers, the practical work was an 
essential part of the curriculum, and they would have had to cover the unit in some other way, 
which they felt would not have been as effective. The teachers mentioned the stimulus provided 
by an additional voice to complement their own work, especially when the information was from 
an expert in the field. Books and films would not bring the same reality. A number would not 
have done the unit, if they had not been able to make the visit. 

Negative Comments 

Two teachers had had to do the visit at a time before they covered the topic in class, due to the 
provider s heavy bookings schedule. Some would have liked more communication prior to the 
visit, and others were disappointed in the presentation, especially when it was not pitched at the 
level of younger students, and when the students did not have time to ask questions. 

Ability of the Programme To Meet the Requirements of the Teacher and 
Class 

Thirty-four of the 40 teachers said that the programmes met their needs “very well”, and another 
3 said they met their needs “well”. Only 3 said that the programmes had met their needs “not 
very well”. 

Positive Comments 

Many of the comments were similar to those given in response to the previous question. Others 
praised the presentation, appreciating the presenter’s ability to reach the students’ level and to 
establish rapport with them. They felt that the teachers had been flexible in their approach, 
modifying their presentation in order to fit with the students’ abilities. They said that the written 
materials and the preparation for the visit were of a high quality. Some teachers mentioned that 
the resource had been used widely throughout the school. 




8 



17 



The teachers also talked about the effect of the programmes on the students. The students 
were focused throughout the programme and their enthusiasm was shown through their thank- 
you notes and through the way in which they continued to talk about the visit. Some students 
had said that the visit was the highlight of their year. One teacher said that he had a wide range 
of students, all of whom had been able to achieve something during the visit. 

Some of the particular programmes were seen as meeting special needs very well. The ESOL 
students were said to have benefited greatly from “turning words into reality”. The art pro- 
grammes were described as beneficial in that they raised the cultural awareness of the students as 
well as that of the parents who provided transport. It was felt some might not have otherwise 
visited a gallery. Other programmes were praised for the way in which they linked different 
subjects: for example, the display of art as science, and the links between poetry and painting. 

Negative Comments 

Most of the negative comments were similar to the previous negative responses. The teachers of 
younger students were more likely to have been disappointed in the programmes. Some teachers 
felt that the programmes should have had more preparation in that they lacked direction. One 
was disappointed in the lack of “hands-on” experience. 

Additional Comments 

Most of the additional comments were positive. Twenty-one teachers said that the visit was 
extremely worthwhile, and that the facility was a valuable asset to their region; they intended 
going back again. Many singled out the presenters for special praise, mentioning their expertise, 
helpfulness, flexibility, and organisational skills. A special needs teacher had writte n to all of her 
special needs colleagues in the region, suggesting that they should visit the facility. 

One teacher said that the facility was so popular that she could not always get in to a relevant 
exhibition. Two teachers felt that the facilities were more useful for senior classes. 
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FINANCIAL COSTS 
AND OTHER POTENTIAL PROBLEMS 



Costs 

1— f ^ of the providers charged very low costs (up to $3 per student), or none at all. Most of 
-L -Lthe others were in the $3-$6 range, and 3 were between S6-S15- Four providers charged 
higher fees, 3 to cover the costs of overnight accommodation for 2 or 3 nights, or for a weekend 
workshop. 

Most teachers said that the costs did not cause any problems, since the school was prepared 
to subsidise any students who could not pay. Two schools paid for the visits from school activity 
fees and another covered costs by fundraising. Parents often provided transport, especially for 
the lower level classes. The more expensive trips had been planned well in advance, so that the 
students had time to save the money, sometimes paying in instalments. A number of the lower 
decile schools said that the trip was their major excursion for the year. 

Other Problems 

Few teachers reported other problems. The length of time taken for the trip was sometimes an 
issue for secondary school teachers, who could not fit the visit within a 1-hour period and had to 
make teaching arrangements for their other classes. They had to balance the worth of the visit 
against other demands of the curriculum for the class, or the needs of other classes. One secondary 
school teacher explained: 



One problem is teaching in 1-hour periods, which doesn’t fit in with outside visits. I have to 
J u Igl e my other classes. Also we have lost time from core subjects, as the curriculum was 
expanded through the introduction of subjects like health and technology. There is less time for 
trips. 



Two teachers of junior classes mentioned the difficulty of getting parking close to the facility, 
causing long walks for 5-year-olds. Another would have liked a place for the students to eat lunch. 



SUMMARY AND CONCLUSIONS 



Summary 

C chools from throughout New Zealand participate in LEOTC programmes. The students are 
'-'from all levels of the schools; our sample had more students from the middle years. The 
programmes we evaluated cover a wide range of the curriculum, with greater emphasis on science, 
art, and social science topics. 

In general, teachers are highly appreciative of the programmes, seeing them as a valuable 
resource, which could not have been replicated at the school. Most programmes fit in very 
closely with curriculum requirements and the standard of presentation was seen as very high. 

Financial costs did not appear to pose a major problem, but time was a limiting factor for 
some teachers. The geographic location of the facilities caused some problems for schools that 
are at a distance from the main centres. 

Half of the teachers would have liked a larger range of programmes, including those which 
came to the schools, and more for younger students. 

Conclusions 

The LEOTC programmes in this evaluation provide a valuable service to a wide range of schools 
and classes. Teachers see a need for more programmes, a wider range of facilities, and more 
information about those which are currently available. 

Where problems were identified, they were specific to particular groups, rather than being 
endemic to the LEOTC programmes. Most could probably be solved through feedback to the 
providers, with appropriate corrective action steps. 
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DOCUMENT ANALYSIS 



Background 

HTne document analysis part of the study was intended to address the question about the types 
-L of schools which were participating in the LEOTC programmes. The 5 Auckland providers 
were chosen as the sample since the greater Auckland region covered the full range of school types 
and socioeconomic deciles. It was therefore likely that the pattern of usage for the Auckland 
providers would be similar to that for the rest of the population. 

Information about schools which had participated in the programmes was requested from the 
5 Auckland providers to cover the period from April to the end of July 1997. In some cases this 
corresponded with the most recent milestone report. The material was analysed by levels of 
schools. Unfortunately there was not sufficient information to determine which class years 
participated in the programmes. 

Results 

Primary Schools 

Some of the schools were full primary schools, covering years 1-8, and the others were 
contributing schools, years 1-6. A total of 176 primary schools participated in the programmes. 
Some visited a number of providers and many made more than one visit to the same provider, 
possibly with different groups. They covered all SES deciles, but the highest representation was 
from schools in deciles 1 and 10, as shown in figure 3. 

Three-quarters of the schools were located within the greater Auckland region. Most of the 



Figure 3: Decile of Primary Schools 




Deciles 



others were from surrounding districts, such as Franklin, Rodney, Kaipara, and Hauraki. 
However, schools from the Waikato, Bay of Plenty, Gisborne, and Rangitikei had also travelled 
to Auckland to participate in the programmes. 



Curriculum areas covered were: 

• science: the living world, earth and beyond, rocks and fossils, birds, and spiders; 

• social science: Maori arts and crafts, Maori history, change— life in early Auckland, and 
pioneers of the air; 

• technology: early technology, and special effects (film); 

• language: oral, written, and visual language; 

• mathematics: number, measurement, statistics, and geometry. 

Intermediate Schools 

Groups from 29 intermediate schools visited LEOTC programmes. They also covered the full 
decile range, and most came from the greater Auckland region and its surrounding districts. One 
group came from a Dunedin school. 

The intermediate students covered the same curriculum areas as those listed above for primary 
students. Most providers had organised their topics to cover a range of levels. 

Secondary Schools 

Eighty-six secondary schools took part in LEOTC programmes. Over two-thirds of the schools 
were in the lower half of the decile range, which was a different pattern from that shown by the 
primary and intermediate schools. The schools were mainly from the Auckland region, and its 
surrounding districts, with 16 from other districts in the upper half of the North Island. 

The providers covered a wide range of topics in the secondary curriculum: 

• science: classification, biology, environmental studies, astrophysics and astronomy, earth 
science, and evolution; 

• social science: environmental studies, geography, New Zealand history, and Maori 
studies; 

• art: ceramics, photography, and exhibition design; 

• language: oral, written, and visual language; 

• technology: early technology, and special effects (film); 

• mathematics: statistics and geometry; 

• outdoor education. 

Other Types of Schools 

Other schools participating in the programmes included 3 special needs schools within the 
Auckland region, 1 area school from North Waikato, and 5 kura kaupapa schools from the 
Auckland region. There were also 26 private schools, 1 most of which were small primary schools 
from the Auckland region. 



The private schools included a number of schools which are designated as Christian schools, some that 
also appear to have arisen from the provisions that permit private schools to be registered if they meet 
certain criteria, and some independent schools. 
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Conclusions 



It appears that a wide range of schools participate in the LEOTC programmes organised by the 
5 Auckland providers. Among them, they cover all SES decile groups, but the secondary schools 
are more likely to be in the lower part of the range. The schools are from all parts of the greater 
Auckland region, with about a third located in districts neighbouring greater Auckland and othe r 
parts of the upper North Island. The programmes meet a range of curriculum needs, with a 
greater emphasis on science and social science units. 



OVERALL CONCLUSIONS 



— The results of the 2 aspects of the study show that LEOTC programmes are playing an 
essential role in complementing the classroom work of the teachers. The teachers report that 
they find them extremely valuable, especially in the way that they afford opportunities for 
hands-on” experience. The programmes are planned with curriculum units in mind. It is 
noteworthy that the providers were able to adapt the programmes in order to meet, for 
example, the special needs of children with disabilities, or those with English as a second 
language. 

□ The few criticisms were very specific, and did not detract from the overall worth of the 
programmes. A number of schools, especially those in rural areas, expressed a need for a 
wider range of programmes, including more for younger children, and more that could be 
brought to the schools. 

□ All types of schools participate in the programmes, regardless of socioeconomic ranking, or 
geographic location. 

□ If the provision were to be expanded, there appears to be a need for more programmes in the 
curriculum areas of language, technology, and mathematics. 

□ Teachers would appreciate receiving more information about the programmes, possibly in 
the form of a catalogue produced annually by the Ministry of Education. 
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APPENDIX 



Questionnaire for Telephone Survey 

Introduction: 

The Ministry of Education has commissioned NZCER to carry out an independent evaluation 
of the LEOTC programme. Additional funding has recently been granted to the programme and 
the Ministry wants to be sure that the programmes are meeting the curriculum needs of the 
schools. 

According to our records, your school/class has taken part in a programme provided by 
. I would like to talk to you about it. 

Question 1: What parts of the curriculum did the programme fit in with ? What units of work, or 

modules? 



Question 2: What did the (provider) do in order to cover this unit/module? 

(i) Before the visit ? 



(li) During the visit 



(tti) After the visit, as a follow-up ? 



[Prompts]: Written materials 

Talks A discussions 
Other resources: slides, videos etc 
Anything else 

Question 3: How important was this visit/programme for the coverage of the unit/module? Could 

you have managed without it? Yes/No 

If you hadn’t been able to do it, what would you have done? 
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Question 4: 



Question 5: 



Question 6: 



Question 7: 



Question 8: 



How well did the programme meet your requirements. Was it very well, well, not very 
well, or poorly? 



What was that? 



What costs were involved for your class? 



(i) [If any] Did this cause any problems for you? Yes/No 

(ii) [If yes] How did you cope? 



Were there any other problems with your participation in the programme? Yes/No 
[If yes] What were they? 



Is there anything else you would like to say about the programme? 



Thinking generally about your needs in order to cover the curriculum/curricula, do 
you have access to enough outside providers? Yes/No 

[If no] What else would by useful? 
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